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Abstract 

(N : 

,__( ^ In his Ph.D. disseration, Feldman and his collaborators define the linear programming decoder for binary linear 

■ codes, which is a linear programming relaxation of the maximum-likelihood decoding problem. This decoder does 
I not, in general, attain maximum-likelihood performance; however, the source of this discrepancy is known to be the 

Q ■ presence of non-integral extreme points (vertices) within the fundamental polytope, vectors which are also called 

^ [ nontrivial linear programming pseudocodewords. Restricting to the class of cycle codes, we provide necessary 

■ conditions for a vector to be a linear programming pseudocodeword. In particular, the components of any such 
I pseudocodeword can only assume values of zero, one-half, or one. 

q'. I. Introduction 

^ ■ Let C be a binary linear code of length n, and consider transmission over a binary-input, memoryless 
channel. Under the the additional assumptions that the channel is symmetric and that each codeword is 
equally likely to be transmitted, a maximum-likelihood decoder returns a codeword that minimizes the 
probability of word-error. It is well-known (see, e.g., [T], fi\, f8l|) that the set of maximum- likelihood 
(ML) codewords coincides with the solution set of the following integer program: 



> 

m 



minimize A^c 



(1) 

subject to c G C 



where A is the vector of log-likelihood ratios based on the channel output. While solutions to integer linear 
CsJ programs such as Problem ([T) are often of great interest, solving integer linear programs in general is 
psj ] quite difficult. On the other hand, solutions to linear programs (problems in which the output vector is not 
constrained to be an integer vector) can often be found quickly (using, e.g., the simplex algorithm), and 
they can even be found in polynomial time (using, e.g., the ellipsoid method) [5J. A common approach 
to solving integer linear programs therefore is to solve a related linear program (or a series of linear 
"H programs) and then interpret the solution in light of the original problem. 

rN Letting conv(C) be the convex hull of C in M" (where we regard codewords as vectors in {0,1}" C M"), 
c3 Problem ([T]) is equivalent to 

minimize A^f 
subject to f G conv(C). 

Problem (|2l) is a linear program; however, the constraints needed to define the feasible region conv(C) 
explicitly are likely to be either (a) difficult to compute or (b) very large in number. Elaborating on this, 
suppose that there was a polynomial time algorithm that was capable of taking a general binary linear 
code C and producing a set of constraints describing conv(C). This algorithm, along with the ellipsoid 
method, can then be used to solve instances of the COSET WEIGHTS decision problem (see [4] for a 
definition) in polynomial time. Since COSET WEIGHTS is NP-complete [H, this would imply P = NP 



This work was supported in part by NSF grants DMS-0602332 and DMS-0838463. 

N. Axvig is with the Department of Applied Mathematics, Virginia Military Institute, Lexington, VA 24450, USA. 
N. Axvig's email: axvigndlO@vmi.edu 



2 



In an attempt to find solutions to Problem © efficiently, Feldman Q defines the linear programming 
(LP) decoder. The linear programming decoder returns as its output the solution to a linear programming 
relaxation of Problem Q, namely 

minimize A^f 

subject to f G P. ^ ' 

The feasible set V of this linear program is known as the fundamental polytope. The fundamental polytope 
has several properties that make it a reasonable choice for a relaxation of Problem not the least of 
which is that, from a computational complexity standpoint, it is easier to describe than conv(C). More 
specifically, if we assume that the row-weight of the parity-check matrix defining the code is bounded by 
a constant, then the number of constraints that describe V is bounded by a polynomial in n, where n is 
the length of the code (see Section ITl-CI) . 

The fundamental polytope is a subset of the unit hypercube [0,1]" and thus is bounded. Because V is 
bounded, the general theory of linear programming states that (a) a solution to Problem © always exists 
and that (b) we may assume that at least one solution is an extreme poinlQ of the underlying polytope 
We therefore make the convention that the LP decoder always retums an extreme point of the fundamental 
polytope. As such, we say that any extreme point of P is a linear programming (LP) pseudocodeword. 
Linear programming pseudocodewords are the principal objects of investigation in this paper. 

In [71, it shown that the set of integer extreme points of V is precisely the set of codewords C. Such 
integer vectors are therefore called trivial linear programming (LP) pseudocodewords. Feldman [7 J uses 
this relationship between C and V to show that the LP decoder has the ML-certificate property: if an 
optimal solution f * to Problem (|3]) is a integer vector, then f * must be an ML codeword. It is often the case, 
however, that the fundamental polytope contains non-integer extreme points, known as nontrivial linear 
programming (LP) pseudocodewords, in addition to the integer- valued codewords. By the ML-certificate 
property, it is precisely the presence of nontrivial LP pseudocodewords that prevents the LP decoder from 
attaining the performance (with respect to word-error rate) of the ML decoder. 

In this paper, we provide new necessary conditions for a vector to be a linear programming pseu- 
docodeword of a cycle code. In particular, we show that any LP pseudocodeword of a cycle code must 
be half-integral; that is, any LP pseudocodeword must be an element of {0, |, 1}". Moreover, in proving 
this half-integrality we also discover that LP pseudocodewords of cycle codes display additional structure 
that can be stated succinctly in terms of the code's Tanner graph. The results of this paper are applied 
in Bill, where the author and his collaborator provide complete graphical characterizations of both LP 
pseudocodewords and minimal linear programming pseudocodewords (see L9J) for the class of cycle 
codes. 

The remainder of this paper is organized as follows. Section |nl the background section, is broken into 
three parts. In Section III-A[ we introduce relevant coding theory terminology and the family of cycle 
codes. Section ITl-BI reviews some basic definitions and facts about extreme points, and Section ITl-CI gives 
an explicit description of the fundamental polytope. Section UlI] makes up the mathematical bulk of this 
paper, and it is divided into two parts. Section ITlI-AI is dedicated to proving a technical result that is then 
used in Section ITlI-B I to prove Theorem IIII.l 1[ which is the main result of this work. We make concluding 
remarks in Section |IV] 

II. Background 

A. Background on Coding Theory 

The results herein require knowledge not only of a code itself but also of the specific parity-check 
presenting the code. By "code," we therefore mean a binary linear code equipped with a fixed parity- 

'Extreme points are often referred to as vertices in linear programming literature. We refrain from applying the term "vertex" to polyhedra 
to avoid confusion when speaking of both graphs and polyhedra simultaneously. Thus, the terms "node" and "vertex" are reserved for graphs, 
and "extreme point" is reserved for polyhedra. 
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check matrix H 

Definition II.l. A Tanner graph is a bipartite graph G with vertex set XUU and edge set E. The elements 
of X are called variable nodes, and the elements of U are called check nodes. 

Given an r x n parity-check matrix H = (hj^i), one can construct the Tanner graph G = G{H) of H 
as follows: set X = {xi,X2, . . . , x„} and U = {iti, 1x2, . . . Ur}. Define G = G{H) to be the graph whose 
vertex set h X VMJ with Xj adjacent to Uj if and only if hj i = 1. In other words, G{H) is the is the 
bipartite graph whose bipartite adjacency matrix is H. Conversely, one can derive a parity-check matrix 
from a Tanner graph: index rows by vertices in U and columns by vertices in X. Set h^^x = 1 precisely 
when X and u are adjacent in G. In this manner one can see that there is a bijective correspondence 
between Tanner graphs and parity-check matrices. 

The Tanner graph is the structure on which the class of message-passing decoding algorithms, e.g., the 
min-sum algorithm or the sum-product algorithm, operate. Loosely speaking, message-passing decoding 
algorithms work by relaying bit and/or reliability information across edges of the Tanner graph in an 
iterative fashion. For the purposes of this paper, however, the Tanner graph is introduced to provide a 
graphical environment in which to view codewords. 

Definition II.2. Let G = {X U U, E) be a Tanner graph, and let f be an assignment of binary values to 
the variable nodes of G. The assignment f is called a valid configuration on G provided that for each 
u E U the quantity J2xeN{u) '■^ ^ven, where, as is standard, N{u) denotes the neighborhood ofu in G, 
i.e., the set of all x E X that are adjacent to u. 

By identifying valid configurations on a Tanner graph with vectors in {0, 1}", we obtain a natural 
correspondence between valid configurations and codewords: 

Proposition II.3 (see, e.g., [3]). Let C be a code with parity-check matrix H, and let G = G{H) = 
{XUU, E) be the corresponding Tanner graph. The set of valid configurations on G corresponds precisely 
to the set C of codewords. 

By Proposition |II.3[ codewords of G may be viewed as valid binary configurations on a Tanner graph. 
This graphical realization of codewords is further displayed in the family of cycle codes. 

Definition II.4. A cycle code is a code G equipped with a parity-check matrix H that has uniform column 
weight 2. 

We focus on cycle codes because they provide for a more fruitful analysis than general codes. This is 
due to their special structure: let C be a cycle code with Tanner graph G. If c G G, Proposition III. 31 states 
c must be a valid configuration on G. Since every variable node of G has degree two, the subgraph of 
G induced by supp(c) U A^(supp(c)) is even, i.e., all nodes in the subgraph have even degree (herein, 
supp(v) is used to denote the support of v: the set of positions where v is not zero). Thus, the subgraph 
induced by supp(c) U A^(supp(c)) must be a union of edge-disjoint simple cycles. The converse is also 
true: the indicator vector for a set of variable nodes within a union of edge-disjoint simple cycles is a 
codeword. 

Example II.5. The parity-check matrix 

0" 



1 

1 
1 1. 

^This is a departure in terminology from many standard texts and publications in coding theory where a code is described intrinsically as 
a subspace, not extrinsically as the kernel of a specific matrix. 
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Fig. 1. The configuration corresponding to tlie codeword c — (1, 1, 0, 0, 1, 1, 0, 1, 0) of Example III. 5 



has uniform column weight two, and hence defines a cycle code. One can check that the vector 

c=(l,l, 0,0,1,1, 0,1,0)^ 

is in the null-space of H. It also gives rise to a simple cycle in the Tanner graph of the code; see Figure [7] 
for an illustration. We note that an isomorphic Tanner graph appears in Figure 6. 1 of [10]. 



B. Background on Linear Programming and Extreme Points 

Most of the material in this section can be found in [SJ. For our purposes, we define a linear program 
to be an optimization problem of the form 



minimize 
subject to 
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ap > h 
aj^f < h, 
ajf = 6, 
/. >0 
/. <0 



3 e M2 
3 e Ms 

i G A^i 



where q, f, and the a^'s are n-dimensional real vectors and the index sets Mi, M2, M3, A'^i, and N2 are 
finiteJl The vector q is called the cost function of the linear program. The linear functional q^f is the 
objective function or the cost function. The vectors a^ are called constraint vectors, or simply constraints. 
Also called constraints are the (in)equalities of the form ajf > 6j, ajf < 6j, or ajf = hj. 

Any vector f that satisfies all of the linear program's constraints is called & feasible solution, and the 
set of all feasible solutions is the feasible set. A feasible solution that minimizes the objective function 
is an optimal feasible solution, or simply an optimal solution. Note that while the optimality of a vector 
f depends on both the constraints and the cost function, the feasibility of a solution f depends only on 
the constraints. Since the feasible set for a linear program is an intersection of closed half-spaces in M", 
it is a polyhedron. A polyhedron that is also bounded is called a poly tope. When dealing with polyhedra 
in the context of linear programming, particular attention is paid to the extreme points. 



^In general, a linear program can be either a maximization or a minimization problem. There is, however, no essential loss of generality 
in using this convention: maximizing q^f is equivalent to minimizing (— q)^f. 
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Definition II.6 ((5], Definition 2.6). Let M. be a nonempty polyhedron in W^. A point uj E Ai is an 
extreme point provided that it is not in the convex hull of Ai \ {u:}. In other words, to is an extreme 
point of M. if it cannot be written as a convex sum ay + (1 — a)z for y, z G \ {uj} and a G (0, 1). 

The importance of extreme points is summarized in the next two theorems. 

Theorem II.7 ([5|, Theorem 2.7). Suppose that a linear program has feasible set M.. If M. has an extreme 
point and an optimal solution to the linear program exists, then there exists an optimal solution that is 
an extreme point of M.. 

Theorem III.7I implies that we may assume that an optimal solution to a linear program occurs at an 
extreme point. In particular, when solving a linear program whose underlying polyhedron is bounded, 
an optimal solution is guaranteed to exist and it therefore suffices to consider only extreme points as 
candidate solutions. On the other hand, given an extreme point e of a polyhedron M. there is always 
some linear program with M. as its feasible set whose unique solution is e: 

Theorem II.8 ([HI, Theorem 2.3). Let M. be a polyhedron in M". If uj is an extreme point of A4, then 
there exists a cost function q G M" such that uj is the unique optimal solution to the following linear 
program: 

minimize q^f 
subject to f G A^. 

In other words, c^u < q^f for all i E Ai\ {a;}. 

It is often convenient to represent extreme points algebraically instead of geometrically. We say that 
a constraint of the form a-^f > b, a^i < b, or a^f = 6 is active at uj if a^oj = b, i.e., if the constraint 
is met with equality. If the polyhedron Ai is defined by linear equality and inequality constraints and uj 
is an element of M", then we say that w is a basic solution if all equality constraints of Ai are satisfied 
and the set of active constraint vectors spans all of M". If is a basic solution that satisfies all of the 
constraints, it is called a basic feasible solution. Using these algebraic notions. Theorem III.9I gives an 
altemate characterization of extreme points. 

Theorem II.9 ([m. Theorem 2.3). Let Ai be a nonempty polyhedron in M" defined by a given set of 
linear equality and inequality constraints, and let uj E Ai. The following are equivalent: 

(a) UJ is an extreme point of Ai, 

(b) UJ is a basic feasible solution of Ai. 

Theorem III.9I relates a geometric invariant to the specific algebraic representation of a polyhedron. It 
also points to a method of finding extreme points: suppose that ai, . . . , a„ is a set of n linearly independent 
constraint vectors, and let 6i, . . . , 6„ be the associated constraint values. Let A be the matrix whose rows 
are ai, . . . , a„, and let b be a column vector whose entries are 6i, . . . , 6„. Since the rows of A are linearly 
independent, there is a solution to Ai = b, namely f = A~^h. By the construction of f, it satisfies the 
constraints ai , . . . , a„ with equality. If f turns out to satisfy the rest of the constraints of the polyhedron, 
then f is a basic feasible solution and hence, by Theorem |II.9[ an extreme point. 

Conversely, if f is an extreme point of a polyhedron, then Theorem III. 9 1 implies that there exists a set of 
n linearly independent constraint vectors that are active at f; call them ai, . . . , a„. With A and b as in the 
preceding paragraph, the extreme point f must be the unique solution to Ai = h. This interpretation of an 
extreme point as a solution to a linear system of equations gives the proof of the following well-known 
theorem. 

Theorem 11.10. Let Ai be a non-empty polyhedron in M". If every constraint vector defining Ai has 
rational coefficients, then every extreme point of Ai is a rational vector 
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C. The Fundamental Polytope 

We now explicitly describe the fundamental polytope, which is the feasible region over which Feldman's 
linear programming decoder operates (see Problem (|3)). 

Definition 11.11 ([7]). Let C be a code with parity-check matrix H and Tanner graph G = (X U f/, E). 
The fundamental polytope V = V{H) is the set of all vectors f G M'"^' satisfying the following constraints: 

(a) 0</j^<l for all x E X, and 

(b) fx + (1 — fx) < ^ 1 for all pairs (u, S), where u E U and S is a subset of N{u) 

xeS x£N{u)\S 

with odd cardinality. 

Feldman Q offers a heuristic justification for the constraints defining V, which we reproduce here for 
the sake of clarity. Since all codewords lie in {0, 1}", the components of a vector in V should not be able 
to assume values outside of [0, 1], but since linear programs require convexity we permit these variables 
to assume intermediate values. The other nontrivial family of constraints is best explained in the integer 
case, for then these constraints can be viewed as cutting planes which forbid "bad" configurations. To 
be explicit, suppose that f G {0,1}" is not a codeword. This means that there exists a row of the 
parity-check matrix H such that r ^ ■ f = 1, which in turn implies that there is a check node j E J of the 
Tanner graph T such that f assigns an odd number of ones to the variable nodes of N{j) in T. Letting 
S denote this subset, we see that f is such that 

= \S\ + \N{j)\S\ 

= mj)\ 

>\Nij)\-l. 

Hence, the non-codeword integer vector f is excluded from V. Conversely, if f G {0, 1}" is a codeword, 
then one can check that it satisfies every constraint of the fundamental polytope. These two results are 
summarized in Theorem III. 121 below. 

Theorem 11.12 (Q). Let C be a code with parity-check matrix H and fundamental polytope V = V{H). 
An integer vector f is an element of V if and only if f E C. 

Since all the constraints of the fundamental polytope are integer vectors. Theorem III. 101 states that all 
extreme points of the fundamental polytope must be rational vectors. If an extreme point f is an integer 
vector. Theorem III. 121 states that f must be a codeword. As mentioned in the introduction, there are often 
fractional extreme points of V as well - see Example III. 131 In Section Hill we investigate the structure of 
the non-integral extreme points. 

Example 11.13. Let C be the cycle code whose parity -check matrix is given in Example 1/7. 51 and whose 
Tanner graph is depicted in Figure [7] and let V be the fundamental polytope of C. Theorem \II.12\ states 
that the set of integer extreme points of V is precisely C. Since the dimension of C is k = 4, it follows 
that V contains 2^ = 16 integer extreme points. As is shown in [2}] (and also in /17]/j, there are exactly 
four non-integral extreme points of V: 

fi i 1 1 i i 

V2' 2' 2' -^i"'"' 2' 2' 2 J 
V2'2'2''''2'2'2/ 

fi 1 1 1 i i 

V2'2'2''''2'2'2/ 

fi 1 1 1 1 1 i 1 1) 

V 2 ' 2 ' 2 ' ' ' ' 2 ' 2 ' 2/' 
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III. LP PSEUDOCODEWORDS OF CYCLE CODES ARE HALF-INTEGRAL 

This section is dedicated to proving that linear programming pseudocodewords are half-integral. Along 
the way, we will see that LP pseudocodewords also have a close relationship with unions of vertex- 
disjoint simple cycles in the Tanner graph. This additional structure turns out to be inseparable from 
half-integrality. We make this latter fact explicit in Theorem IIILl 11 which is the main result of this paper. 

In order to prove Theorem IIILl 11 we adopt the viewpoint suggested by Theorem III.9I - namely, we 
view an extreme point of a poly tope as a solution to a linear system of equations. In our case, the 
equations forming this system come from the constraint vectors of the fundamental polytope that are 
satisfied with equality. We proceed as follows. We begin by prove a technical lemma pertaining to active 
constraints in Section IIII-Al This lemma is then applied to cycle code^ in Section IIII-Bl giving a proof 
of Theorem IIILl 11 



A. A Technical Lemma on Active Constraints 

Let C be a code defined by parity-check matrix H, and let G = {X U U, E) be its Tanner graph. 
We may rearrange the constraints defining the fundemental polytope to put them into the standard form 
presented in Section III-Bl 

/,< 1 VxeX (4) 

> yxex (5) 

Yl l-^l"^ such that 1^1 is odd 



xGS xeN{u)\S 



For convenience, we use the notation {u, S) to represent the constraint vector of V corresponding to the 
check u and the odd-sized subset S C N(u). 

Definition III.l. Let C be a code presented by a parity-check matrix H. Let G = {X U U, E) be its 
Tanner graph, and let V be its fundamental polytope. For a check node u E U, a subset S C N{u), and 
a vector f G [0,1]", the cost of S at u relative to f is given by 

xeS x(^N{u)\S 

If there exists an odd-sized subset S C N{u) such that /«„,f(S') = |A^(m)| — 1, then we say that the check 
u is active at f and that the set S is active for f at u. 

Note that a vector f G [0,1]" is in the fundamental polytope if and only if /Cn,f(<S') < |A^(ii)| — 1 
for all pairs {u,S) such that S is an odd-sized subset of N{u). The next lemma follows directly from 
Definition lIII.ll in conjunction with the definition of the fundamental polytope. 

Lemma III.2. Let G be a code presented by a parity-check matrix H. Let G = {X U U, E) be its Tanner 
graph, and let V be its fundamental polytope. If u E U is active at some f G P and a G (0, 1), then f 
can assume the value of a in at most max{^, jz^} positions of N{u). In particular, f can assume the 
value of I at most twice in N{u). 

Proof: Let D C X be the set of variable nodes in N{u) that are assigned a value of a by f, and let 
d := \D\. Since u is active at f, there exists an odd-sized subset S C N{u) such that S is active for f at 
u. We consider two cases. 

If a > |, then a> 1 — a. We therefore have 

\N{u)\ - 1 = K,,f(5) < \N{u)\ - d{l - a), 



''while this paper's main result concerns linear programming pseudocodewords of cycle codes, Lemma llll.3l can be applied to any binary 
linear code. 
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which implies that d < = maxj-, -r—}- 

On the other hand, if a < |, then a < 1 — a. We therefore have 

\N{u)\-l = K^,,{S)<\N{u)\-da, 

which implies that d < - = maxj-, -r—}- ■ 
Recalling that the symmetric difference between two sets Si and 5*2 is 5*1 A 5*2 := (iSi U 5*2) \ (5*1 fl 5*2), 
we now prove Lemma IIIL3[ which is crucial in establishing Theorem llll.lli 

Lemma III.3. Let C be a code presented by a parity-check matrix H. Let G = {X U U, E) be its Tanner 
graph, and let V be its fundamental polytope. Fix f G P and u E U. If u is active at f and f is not 
integral on N{u) (i.e., there is some x G N{u) such that < /^^ < Ij, then there are at most two active 
sets for f at u. 

Moreover, in the case that the number of active sets for i at u is exactly two, the symmetric difference 
of these two sets consists of exactly two variable nodes, and these two variable nodes are exactly the 
neighbors of u at which f is not integral. 

Proof: Let f G "P and u E U he, given such that f^(") is not integral and u is active at f . For each 
X G N{u), the value fx appears in the expression for K„,f (S") as if a; G 5" or as 1 — fx if x G N{u) \ S. 
The basic idea of this proof is to address the following question: how can make the largest contribution 
to Ku,i{S)l It is clear that J^. > 1 — fx if and only if fx> \ and that < 1 — fx if and only if fx < |. 
From this, we see that Ku,f(<S') is maximized exactly when S consists of all nodes x such that fx > |, 
and possibly some nodes x with = i. In a search for sets S that are active for f at u, however, we 
can only consider those sets 5* whose cardinality is odd. This parity-based issue is highly dependent on 
f itself, about which we know little. We therefore consider several cases, each of which takes the greedy 
solutions that ignore parity and mashes them into solutions that respect this parity condition. 

Define the following sets: 

. £ := {x G N{u) : fx<h} 

. S:={xe N{u) : fx = |} 

. g:={xe N{u) : fx > |} 

. Q:={xe N{u) : |i - fx\ < || - fx'\ for all x' G N{u)}. 
The set Q contains all nodes x such that fx is closest to ^ among all x' G N{u); note that Q = £ if 

Since u is active at f, by Lemma IIII.2I we know that < < 2. Suppose first that \£\ = 2, and write 
Q = £ = {qi,q2}- If \G\ is odd, then the sets 5*1 := Q and 6*2 := ^ U Q are the only two odd-sized 
subsets of N(u) that maximize Hu^t over all odd-sized subsets of N(u) . Moreover, since u is active at f , 
the maximum value achieved by K„ f over all odd-sized subsets of N(u) is precisely |A^(m)| — 1. Having 
\£\ = 2 therefore forces all values on N{u) \ Q to be integral. Since and 5*2 satisfy S'iAS'2 = Q 
and IS'iAS'21 = 2, the lemma is proved in this case. If \Q\ is even, then the sets 5*1 := ^ U {qi} and 
S2 ■= G U {q2} are the only two odd-sized subsets that maximize K„f. As in the previous case, the 
maximum value achieved by K^f over all odd-sized subsets of N{u) is |A^(ti)| — 1, so \£\ = 2 again 
forces all values on N{u) \ Q to be integral. Since S'iAS'2 = Q, we are done in this case as well. 

Now suppose that \£\ = 1. If \Q\ is odd, then S" = ^ is the unique maximizer of «;„ f over all odd-sized 
subsets of N{u). If \Q\ is even, then S" = ^U£^ is the unique maximizer of /c^ f over all odd-sized subsets 
of N{u). In either situation there is at most one active set at u, and so the lemma is proved in this case. 

Finally, assume that \£\ =0. If \Q\ is odd, then S' = ^ is the unique maximizer of f over all odd-sized 
subsets of N{u) and the lemma is proved. If \Q\ is even, then the collection of all active sets for f at w 
is given by {Sq \ q E Q}, where Sq is defined as follows: 




gu{q} ifq^g 

G\{q} ifqeg. 
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If |Q| = 1, there is a unique set Sg that maximizes Hu,fiS) and we are done. So suppose that |Q| > 1, 
and let qi and q2 be distinct elements of Q. By the definition of Q, we have /^^ = f^^ if qi,q2 E C or 
92 e and fq^ = l — fq^ if gi G £ and q2 E G or vice- versa. If qi,q2 E C , then 

|iV(ii)|-l = /t„,f(5,J 

= E + E (1 - 

xeAf(M)\5gi 

= E + + (1 - 
+ E (1 - 

xG7V(«)\(S'giU{g2}) 

= E + E (I-/-)- 

x€Sqj^\{qi} xeN{u)\{SqiU{q2}) 

We conclude that f must be integral on N{u) \ {qi, q2}; otherwise, it would not be possible for k„ f (S^J 
to attain the value |A^(u)| — 1. A similar argument yields the same conclusion in each of the other three 
cases. 

By assumption we have that f must be non-integral in at least one position of N{u), so either < /^^ < 1 
or < < 1. It follows from this and from the definition of Q that Q = {^1,52} and that < /g^ , < 1. 
Thus, there are exactly two active sets Sq^ and Sq^, these sets satisfy ISq^^ASq^] = 2, and Sq-^ASq^ is exactly 
the set of indices where f is not integral. ■ 



B. A Proof of Theorem \III.11\ 

Let C be a code with parity-check matrix H, Tanner graph G = {X UU, E), and fundamental polytope 
V. Let G P be an LP pseudocodeword, i.e., an extreme point of V. By Theorem 111.91 is a basic 
feasible solution. Thus, the span of the constraint vectors of V that are active at uj has dimension n := \X\. 
For each node to which assigns either a or a 1 there is an active constraint of the form dH) or (|5]). This 
set of constraint vectors is linearly independent, so we may extend it by other active constraint vectors, 
which will necessarily be of the form (w, S), to obtain a set of n linearly independent constraint vectors 
that are active at uj. Up to a permutation of rows and columns, we may write these n constraint vectors 
in matrix form as 

In—m 



L. 



where the last m columns represent the variable nodes in J^^^ := {x \ < uj^ < 1} . Note that since the 
last m rows of L^^ come from constraints of the form {u,S), each entry of L^^ is either —1,0, or +L 
Note also that since det(Lt^) = det(i?t^), the fact that L^^ is invertible implies that the square submatrix 
Ri^ is also invertible. 

Since each row of L^^ is a constraint vector that is active at a;, we have that L^^oj = z^^, where z^^ is 
an integer vector determined from the right-hand sides of constraints dH) - (|6l). If we knew that L^^ takes 
entries only from ^Z, we could write u; = L^^z^^ and conclude that a; G {0, |, 1}". Our next goal is to 
show that, in the case of cycle codes, L^^ has entries only in ^Z. Because of the block structure of L^, 
this amounts to showing that R^^ has entries only in |Z. To show this algebraic fact about R^, we turn 
to graphical methods. 

Definition III.4. Let C be a code with parity-check matrix H, Tanner graph G = {X U U, E) and 

fundamental polytope V. Let u be a nontrivial linear programming pseudocodeword, define := {x G 
X I < oja; < 1}, and set m = Let R^^ be an m x m matrix formed as above, and define U,^ to be 
the set of all check nodes u E U such that a constraint of the form {u, S) is represented in the rows of 
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L^j. Define the subgraph Gr^ of G as follows: let the vertex set of Gr^ be U U^, and make x G 
adjacent to u E U,^ if and only if one of the rows of R^^ arising from u has a non-zero entry in the x'^ 
position. 

Lemma III.5 (see also [7], Theorem 7.1). Let C be a code with parity-check matrix H, Tanner graph 
G = {X U U, E) and fundamental polytope V. Fix f eV and u E U. If u is incident to a variable node 
X E J^f := {x E X \ < fx < 1}, then it is incident to at least two such nodes in J-f. 

Proof: Assume that u is incident to one and only one variable node xq E J-f. Then every variable 
node in N{u) \ {xq} is assigned a value of or 1 by f. Let Ou ■= {x E N{u) \ = 1}. If \Ou\ is even, 
then := C„ U {xq} has odd cardinality. Notice that K„,f(S') > IS"] + \N{u) \ S"] - 1 = |A^('u)| - 1, which 
means that f does not satisfy constraint {u, S) of V given by 

KuAs) = J2f-+ E a-f.)<\N{u)\-i. 

xeS x&N{u)\S 

Hence, f is not an element of V. 

If \Ou\ is odd, set S = and observe that Ku,f{S) > \S\ + \N{u) \ S\ - 1 = \N{u) \ - 1. Again, f 
fails to satisfy constraint (tt, S) of V, so f is not an element of V. ■ 

Remark IIL6. Lemma \III.5\ implies that for any i E V, J^f is a stopping set By definition, a stopping 
set is a set of variable nodes V such that if a check u is adjacent to some v E V, then it is adjacent to 
at least two distinct elements of V. Stopping sets are significant in the study of iterative message-passing 
decoding on the binary erasure channel: the belief propagation algorithm fails to decode if and only if 
the set of erased bits contains a stopping set [6^]. 

We now restrict to the case where C is a cycle code. 

Lemma III.7. Let G be a cycle code with Tanner graph G = {X U U, E) and fundamental polytope V. 
Let and R^j be matrices formed as above, and let T,^ and U^, be the subsets of X and U, respectively, 
as given in Definition \III.4\ For any nontrivial linear programming pseudocodeword uj E V we have 
Ucj ^ N{J'^), l^^l = \Ui^\, and Gr^ is 2-regular 



Proof: Let be given as above, so that 

L„, = 



^n—m 

Qu} R 



We first show that C A^(J^^). Let u E 11^^ he given. There must be a corresponding 5" C N(u) such 
that (m, S) is a row of L^^. Since R^^ is non-singular, this row must involve some variable nodes in J^^. 
Thus, u E N{J^^), so C N{J^^). 

We now show that |^7aj| = I-^ojI- This amounts to showing that no check is represented in the rows of 
[Qui Rui] more than once. Clearly, a single constraint vector (u, S) associated with u cannot appear in the 
rows of [Qi^ Ri^] more than once, since otherwise R^^ would not be invertible. 

Suppose now that for some u E U^^ there are two distinct subsets Si, S2 C N{u) such that the constraints 
(m, Si) and {u, S2) are both rows of [Qu; Rc^]. Then both and S2 are active for lh at u. In general, the 
vectors {u, Si) and {u, S2) are such that {u, Si) = —{u, S2) when we restrict to those positions of S'iAS'2, 
and (u, Si) = {u, S2) = on the positions of X\ N(u). By applying Lemma llII.3l to this check node, we 
see that 5*1 A5'2 = {x E N{u) | < < 1}. This means that (u, Si) = -{u, S2) on S'iA5'2 = N{u)r]J'u> 
and (n, ^i) = {u, S2) = on \ N(u). It follows that the projections of (u, Si) and {u, S2) onto the 
positions of are scalar multiples of one another, which contradicts in the fact that R^i is invertible. 
We conclude that |f/tj| = \J-'u>\. 

To prove the third and final claim of the lemma, we bound the number e of edges in Gr^ in two 
different ways. Since C is a cycle code, each variable node has degree exactly 2 in G, so every variable 
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node in Gr^ has degree at most 2. Thus, e < 2| On the other hand, since C N[J^t^), Lemma UlI.SI 
yields e > 2|f/tj|. Since \J^^\ = |f/tj|, the result follows. ■ 

Proposition III.8. Let C be a cycle code with fundamental polytope V. For any nontrivial linear 
programming pseudocodeword uj E V and any corresponding matrix R^, the matrix has entries 
only in { — i, 0, 

Proof: By Definition IIIL41 the non-zero entries of R^^ give the incidence structure of More 
formally, the matrix \Ri^\ obtained by taking the coordinate-wise absolute value of each entry in R^^ is the 
bipartite incidence matrix for Gr^ . By Lemma IIIL71 Gr^ is a 2-regular bipartite graph; therefore, each 
connected component of Gr^ is a cycle of even length. Using this connection between R^^ and Gr^, we 
may assume (by permuting rows and columns) that R^^ is a block diagonal matrix 



Ru 



where each block is square and has the form 



Di 
D2 








D, 



bj 



D 



d2,2 d2,3 





deA 










de-i/-i (i^-i,, 
dfj 



with dp^q E {—1, +1} for all p, q. To show that R^^ has entries only in { — |, 0, +|}, it suffices to show 
that each block D of R^^ is such that D^^ takes entries only in { — |, 0, +|}. In fact, we will show that 
D^^ takes entries only in { — 

Let D he a block of Ru^. Since R^ is invertible, D is also invertible. This implies the existence of 
a unique solution a to the equation Da = Sp, where is the pth standard basis vector. The equation 
Da = Ep gives rise to a set of relations that must hold between the entries of a. 

In the following we take subscripts modulo £ to respect the cyclic nature of D. For all q ^ p, we have 



idq,q+i = 0. Since dg^g+i, dg^g G {—1, +1} we may rearrange to get 



— Og+i, and so 



±aq+i for all q^ p. This in turn implies that all of the entries of a are the same up to sign. 



The equation Da = Sp also implies that apdp^p + ap+idp^p+i 



1. We know from the previous paragraph 



that Op+i is either ap or — Op. Thus, we have to consider two possible equations: ap{dp^p + dp, 
Q'p{dp,p — dp^p^i) = 1. In either case, since (ip,p+i, dp^p G {—1, +1} we have that Op is either 
Combining this with the previous paragraph, we see that all the entries of a are in { — |, +|}- 
Renaming the unique solution to the equation Da = Sp to be ap, we see that 



P+i) 
1 

'2 



= 1 or 
or +i. 







ai a2 . 


• a^ 







D- 



It follows that D^^ takes entries only in { — i, i}. ■ 
The next corollary follows immediately from Proposition IIII.8I and the discussion surrounding the 
introduction of L^^. 

Corollary III.9. Let C be a cycle code with fundamental polytope V. If u) is an extreme point of V and 
Lw is formed from uj as above, then has entries only in |Z. 



We now have the tools to prove the following proposition. 
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Proposition III.IO. Let C be a cycle code of length n with Tanner graph G = {XUU, E) and fundamental 
polytope V. If oj is a nontrivial linear programming pseudocodeword ofV, then uj G {0, |, 1}" and the 
subgraph Gjr^ of G induced by the set = {x | o;^ = |} and its neighborhood N{J^^) is precisely 
Gr^, which is 2-regular 

Proof: Let L^, R^^, and U^^ be given as above. By Corollary |III.9[ has entries in ^Z. By the 
discussion following the introduction of L^^ earlier in this section, we have that L^^uj = z^^, where z^^ 
is an integer vector. Thus, uj = L^^z must have entries that come only from |Z. Since any point in the 
fundamental polytope can only assume values between and 1, a; G {0, ^, 1}". 

Lemma IIII.7I implies that 11^^ C A^(J^^), = |f/tj|, and Gji^ is 2-regular. From U^^ C N^J^^^) and 
Definition lin.4l we see that Gr^ is a subgraph of Gjr^. Letting e denote the number of edges in Gj^^, 
we have that e = 2|J^tj| since C is a cycle code. Lemma IIIL5I implies that 2\N{J'i^)\ < e. Using the fact 
that C N{J^^), we have 

2\U^\ < 2|iV(J-^)| < e = 2|J-^|. 

Since iJ-i^l = \Uuj,\, these inequalities must be tight. Thus Gj^^ has the same number of edges as Gr^ 
and contains Gr^ as a subgraph. We conclude that Gjr^ = Gr^. ■ 
We may now prove the main result of this paper. 

Theorem III.ll. Let G be a cycle code with Tanner graph G = {X U U,E) and fundamental polytope 
V. If tjj &V is a linear programming pseudocodeword of V, then the following two conditions hold: 

(a) u G {0, \, and 

(b) with Ti := {x & X \ = |}, the subgraph T of G induced by TiU N{'H) is 2-regular Equivalently, 
r is a union of vertex-disjoint simple cycles 7i, 72, • • • , 7^- 

Proof: Let a; be a linear programming pseudocodeword of V. If a; is a codeword, the two conditions 
in the statement of the theorem are satisfied and we are done. If, on the other hand, a; is a nontrivial 
linear programming pseudocodeword, we may apply Proposition IIII.IOI to conclude that u) satisfies the 
two conditions. ■ 

IV. Conclusion 

We have shown that a vector u: can be a linear programming pseudocodeword of a cycle code only if a; 
is half-integral. Moreover, the subgraph of the Tanner graph induced by the variable nodes assigned a value 
of I by uj, along with their neighboring check nodes, must be a union of vertex-disjoint simple cycles. 
These necessary conditions, however, are not sufficient to characterize LP pseudocodewords for cycle 
codes. The results from this paper are extended in [2], where the author and his collaborator provide 
complete graphical characterizations of both linear programming pseudocodewords and minimal linear 
programming pseudocodewords (see [9]) for the class of cycle codes. 

Finally, we note that Lemma |III.3l which was an essential ingredient in our proof of Theorem HII. 1 1 1 can 
be applied to any binary linear code. Testing whether this lemma can shed light on the pseudocodeword 
structure of additional families of codes is an object of future pursuit. 
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